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I Backaround: The End to End Performance Tool is used to distinguish performance 
' bottlenecks for select customers. This tool gives objective end- to- end 

{performance metrics of CCO customers' E- commerce transactions. 
It tracks two measurements: (1) the Internet delay from selected CCO customers' 
sites to Cisco, and (2) the IPC (e-commerce) Application delay, and presents this 
data over time . 

I This tool is designed to help identify the exact nature of the response time 

; latency. Currently, the IPCJ-OT (New Ordering Tool) application is being monitored : 

with this tool. However, many applications can be added to this tool and can be | 

used in the same way. 

To our knowledge, there is no tool that is designed to monitor and accurately 
identify customers' performance latencies. The End to End Monitoring tool monitors j 
and represents customers both by customer name and by theater. This tool is unique J; 
in that we can offer our customers additional information about the exact location 
of their performance delays. This information is highly valuable because we can 
pinpoint the time, nature, and location of application vs. Internet latencies. 
This information is, ultimately, going to be used to help identify solutions and ! 
j improve overall customer experience for our customers. | 



1 Possible Prior Art: " " ~ 

[Summary:] The End to End Performance Tool is used to distinguish performance 

" bottlenecks for select customers. This tool gives objective end-to-end 
performance metrics of CCO customers' E-commerce transactions. 
I It tracks two measurements: 

1(1) the Internet delay from selected CCO customers' sites to Cisco, and the 
! (2) Ordering Tool (E-commerce) Application delay 

jllThis data is presented over time. Data is kept on a four week rolling average for 
network latency from CCO to the customer-provided URL and for the internal 
{application being monitored. 

The E2E Tool determines connectivity to the configured customer sites by using a 
| " HTTP HEAD" request to their web server. Alternatively, it may use a simple ICMP 
"ping" to their site. 

This monitoring is performed by the standard EMAN application monitor, and is thus 
has a configurable polling rate and timeout criteria. (Typically sites are polled | 
once every 3 0 seconds) 

i Since both the internet and the CCO servers experience load which varies depending 
on the time of day and day of week, direct comparison of performance of the network j 
i or application's absolute response time is not necessarily useful. This tool 
i therefore develops baselines over a period of weeks to characterize what response 
time is typical for a particular time of day and time of week. It then compares 
measured response times against these baseline values and allows users to see 
periods of both poor performance (absolute response time) and unusually poor 
performance (response time as a percentage of the baseline for that time of day and | 
day of week) . j 

jThe ability to combine absolute measurement of CCO application performance and of 
j network performance between Cisco and its customers against typical values 

(baselines) is key to the usefulness of this tool. This ability sets it apart from ! 
{simple network availability monitoring by enabling users to, at a glance, 
I differentiate "normal" behavior from "abnormal" behavior. 



The secon hase of the End to End tool includes an automated traceroute facility. 
This performs and stores traceroutes in addition to the existing HTTP and ICMP 
monitoring and stores these traceroutes in a database for later retrieval. This 
traceroute data is encoded and ranked based upon several criteriajp 

1 . Hop Count 

2 . Total RTT for all Hops 

3 . Maximum RTT for any Hop 

4 . RTT of the last hop 

5 . Sum of the absolute deviations of the RTTs for each hop 

6. Number of "errors" {no response, host unreachable, etc) 

7. "Traceroute Index" (This is a value which is computed as a weighted product of 
| several of the other metrics and is intended to be a numeric indicator of the 

| "goodness" of a particular traceroute as compared to others to this same 
destination host. It is a relative measure only. 




This data is then linked into the existing response time graphs such that the user 
may click on the graph to view the closest traceroute to that point in time. This 
is intended as a method for users to identify the cause of the degredation which 
| was measured at that time. Traceroutes may be graphed based upon the numeric 
lvalues mentioned above and they may be viewed in detail to show all hops and their 
response times. 

■Traceroutes are also baselined based upon the "traceroute index" mentioned above so 
'that a "representative worst and best case" traceroute is available for each 
baselining interval throughout the week. This is a valuable tool for identifying 
\ in what way a particular period of poor performance is different from one of good 



Restatement: 

; Advantages: Thls solution can not only detect response time latencies, but it can 

— ^also identify the exact location of the performance latency. This data 

| is presented over time. Data is kept on a four week rolling average for network 
| latency from CCO to the customer-provided URL and for the internal application 
being monitored. 

An easy- to- interpret graphical representation shows us whether the Internet 

connection to the customer, or the Application, was performing unusually poorly or 
Swell at any given time. It highlights a difference from the norm, and allows you to 

determine whether the Internet connection, or the Application, or both, were 
| performing unusually worse or better than expected at any given time. Once a 
(problem period is identified, the data may be viewed in several other ways 
1 (absolute network and performance response times, times vs. the baselines, and 

traceroute data) which help in identifying the cause of poor performance for a 
? particular customer. 

(Additionally, no client or monitoring agent installation is required on the 
j customer's computer, server or network. Performance is monitored with the 
customers' complete permission/collaboration and, therefore, the level of 
[imposition on the customer is very minimal. 



; First Written Record URLs: 



j Cisco Use - -'his tool gives objective end-to-end performance metrics of CCO 

— — — — '- cus tomers ' E-commerce transactions. The tool is designed to help j 

Customer Service Tier 1 and 2 support people to identify the source of a customer's 

I CCO problem when a customer has called in with an inquiry or complaint. 

. Future priorities include having a secured version of the tool available to all j| 
Tier 1 Customer Service representatives. This secured version would be an easy-to- 

: interpret representation of whether the problem is an application or Internet 
issue. This will be highly beneficial because customer representatives can 
immediately respond to our customers' inquiries. This will not only reduce costs by 



; reducing the numbers of calls that are escalated to Tier 2 and Tier 3 support, but 
I it will also increase customer satisfaction. 



I Wording Mode!: 



[industry Use' Description of possible uses of the technology by others in the 
~" ~ 1 industry: 



I Public Use:] Tlie P r °duct has not been offered for sale or used outside of Cisco. 

— " We have used this tool, for select customers, but only the core End to End 
team has used this tool. 



: Government Use: " 



I Detectinq Use ' " c '' can regularly review publicly available solutions to determine 
— - similar features are offered. Customer reports of similar offerings 

from other companies may offer visibility into those proprietary solutions. 



Standards:! ~ 



Technologies: 



IP > IP Application Services 

Quality of Service (QoS) > QoS Congestion Management (queueing) 



Networking Solutions: 



Large Enterprise > Networking Solutions for Large Enterprise > Network 
Management Solutions for Large Enterprise 
Large Enterprise > Networking Solutions for Large Enterprise > Content Networking Solutions for Large 
Enterprise 



^Caiegonzaiion Notes: 

Categories Summary 
| [CO] [IN] [NM/NA] 
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PDDs: 



.Supporting Documents: 



Notes: Please contact our team for additional documentation, project plans, 
presentations, and any other information on this tool. 
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